Conformal off-policy prediction

Off-policy evaluation is critical in a number of applications where new policies need to be evaluated offline before online deployment. Most existing methods focus on the expected return, define the target parameter through averaging and provide a point estimator only. In this paper, we develop a novel procedure to produce reliable interval estimators for a target policy’s return starting from any initial state. Our proposal accounts for the variability of the return around its expectation, focuses on the individual effect and offers valid uncertainty quantification. Our main idea lies in designing a pseudo policy that generates subsamples as if they were sampled from the target policy so that existing conformal prediction algorithms are applicable to prediction interval construction. Our methods are justified by theories, synthetic data and real data from short-video platforms

Long-time self-similar asymptotic of the macroscopic quantum models

The unipolar and bipolar macroscopic quantum models derived recently for instance in the area of charge transport are considered in spatial one-dimensional whole space in the present paper. These models consist of nonlinear fourth-order parabolic equation for unipolar case or coupled nonlinear fourth-order parabolic system for bipolar case. We show for the first time the self-similarity property of the macroscopic quantum models in large time. Namely, we show that there exists a unique global strong solution with strictly positive density to the initial value problem of the macroscopic quantum models which tends to a self-similar wave (which is not the exact solution of the models) in large time at an algebraic time-decay rate.Comment: 18 page

Pattern transfer learning for reinforcement learning in order dispatching

Order dispatch is one of the central problems to ridesharing platforms. Recently, value-based reinforcement learning algorithms have shown promising performance to solve this task. However, in real-world applications, the demand-supply system is typically nonstationary over time, posing challenges to reutilizing data generated in different time periods to learn the value function. In this work, motivated by the fact that the relative relationship between the values of some states is largely stable across various environments, we propose a pattern transfer learning framework for value-based reinforcement learning in the order dispatch problem. Our method efficiently captures the value patterns by incorporating a concordance penalty. The superior performance of the proposed method is supported by experiments

simplexreg: an R package for regression analysis of proportional data using the simplex distribution

Outcomes of continuous proportions arise in many applied areas. Such data are typically measured as percentages, rates or proportions confined in the unitary interval. In this paper, the R package simplexreg which provides dispersion model fitting of the simplex distribution is introduced to model such proportional outcomes. The maximum likelihood method and generalized estimating equations techniques are available for parameter estimation in cross-sectional and longitudinal studies, respectively. This paper presents methods and algorithms implemented in the package, including parameter estimation, model checking as well as density, cumulative distribution, quantile and random number generating functions of the simplex distribution. The package is applied to real data sets for illustration

Observation and analysis of diving beetle movements while swimming

The fast swimming speed, flexible cornering, and high propulsion efficiency of diving beetles are primarily achieved by their two powerful hind legs. Unlike other aquatic organisms, such as turtle, jellyfish, fish and frog et al., the diving beetle could complete retreating motion without turning around, and the turning radius is small for this kind of propulsion mode. However, most bionic vehicles have not contained these advantages, the study about this propulsion method is useful for the design of bionic robots. In this paper, the swimming videos of the diving beetle, including forwarding, turning and retreating, were captured by two synchronized high-speed cameras, and were analyzed via SIMI Motion. The analysis results revealed that the swimming speed initially increased quickly to a maximum at 60% of the power stroke, and then decreased. During the power stroke, the diving beetle stretched its tibias and tarsi, the bristles on both sides of which were shaped like paddles, to maximize the cross-sectional areas against the water to achieve the maximum thrust. During the recovery stroke, the diving beetle rotated its tarsi and folded the bristles to minimize the cross-sectional areas to reduce the drag force. For one turning motion (turn right about 90 degrees), it takes only one motion cycle for the diving beetle to complete it. During the retreating motion, the average acceleration was close to 9.8 m/s2 in the first 25 ms. Finally, based on the diving beetle's hind-leg movement pattern, a kinematic model was constructed, and according to this model and the motion data of the joint angles, the motion trajectories of the hind legs were obtained by using MATLAB. Since the advantages of this propulsion method, it may become a new bionic propulsion method, and the motion data and kinematic model of the hind legs will be helpful in the design of bionic underwater unmanned vehicles

Water entry of slender segmented projectile connected by spring

An object that enters the water experiences a large impact acceleration at the initial stage of water entry, which can cause structural damage to objects that are dropped or launched into the water. To reduce the peak impact acceleration, a spring-connected segmented projectile with compressible nose was designed. Through inertial measurement unit and high-speed camera, the influence of the nose compressibility on the initial impact acceleration was qualitatively investigated. The experimental results demonstrate that the introduction of a spring between the nose and the main body of the projectile can significantly suppresses the peak acceleration during the early stage of impact (0–50 ms). Furthermore, the maximum impact acceleration experienced by the main body is only related to the maximum compression of the nose without considering the spring stiffness. In addition, using the spring exerts a slight effect on the non-dimensional pinch-off times of the cavity but increases the initial velocity required for the occurrence of cavity pinch-off events on the side of the main bod

Dynamics and hydrodynamic efficiency of diving beetle while swimming

Diving beetle, an excellent biological prototype for bionic underwater vehicles, can achieve forward swimming, backward swimming, and flexible cornering by swinging its two powerful hind legs. An in-depth study of the propulsion performance of them will contribute to the micro underwater vehicles. In this paper, the kinematic and dynamic parameters, and the hydrodynamic efficiency of the diving beetle are studied by analysis of swimming videos using Motion Capture Technology, combined with CFD simulations. The results show that the hind legs of diving beetle can achieve high propulsion force and low return resistance during one propulsion cycle at both forward and backward swimming modes. The propulsion efficiencies of forward and backward swimming are 0.47 and 0.30, respectively. Although the efficiency of backward swimming is lower, the diving beetle can reach a higher speed in a short time at this mode, which can help it avoid natural enemies. At backward swimming mode, there is a long period of passive swing of hind legs, larger drag exists at higher speed during the recovery stroke, which reduces the propulsion efficiency to a certain extent. Reasonable planning of the swing speed of the hind legs during the power stroke and the recovery stroke can obtain the highest propulsion efficiency of this propulsion method. This work will be useful for the development of a bionic propulsion system of micro underwater vehicle

Effects of eigen and actual frequencies of soft elastic surfaces on droplet rebound from stationary flexible feather vanes

The aim of this paper is to investigate the effect of eigenfrequency and the actual frequency of the elastic surface for the droplet rebound. The elastic surface used in this study is the stationary flexible feather vanes. A fluid-structure interaction (FSI) numerical model is proposed to predict the phenomenon, and later is validated by the experimental that the droplets impact the stationary flexible feather vanes. The effect of mass and stiffness of the surface is analysed. First, the suitable combination of mass and stiffness of the surface will enhance the drop rebound. Second, a small mass system with higher eigenfrequency will decrease the minimum contact time. In the last, the actual frequencies of the elastic surface, approximate at 75 Hz, can accelerate the drop rebound for all cases